Our research is in the area of computational interaction. We use methods of optimization, machine learning, and mathematical modeling to understand and improve our interaction with computers. In particular, our work centers around the following topics:
Optimal adaptation of interfaces
We investigate how to optimally adapt user interfaces at run-time to changes in the usage context, such as the user’s environment, their cognitive load, current task, or personal interaction strategy. On the one hand, this includes formulating optimization models for adapting interfaces. On the other hand, we develop methods to identify changes in the user’s context and models to predict how such changes impact the user’s behavior. Example projects include:
- AdaM: Adapting Multi-User Interfaces for Collaborative Environments in Real-Time. In: SIGCHI Conference on Human Factors in Computing Systems, ACM, 2018.
- Context-Aware Online Adaptation of Mixed Reality Interfaces. In: Symposium on User Interface Software and Technology, ACM, 2019, ISBN: 9781450368162.
- Detecting Relevance during Decision-Making from Eye Movements for UI Adaptation. In: Symposium on Eye Tracking Research and Applications, Association for Computing Machinery, 2020, ISBN: 9781450371339.
- XRgonomics: Facilitating the Creation of Ergonomic 3D Interfaces. In: Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI'21), ACM, 2021.
- AUIT – the Adaptive User Interfaces Toolkit for Designing XR Applications. In: Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology, UIST'22, 2022.
User interaction with intelligent interfaces
Our recent work focuses on modeling and supporting people in their interaction with intelligent interfaces, that is interfaces that make use of machine learning, optimization, or other computational methods to facilitate computer interaction. We strive to make such interfaces comprehensible, predictable and adjustable by end-users. Only then people can efficiently use them and, importantly, will trust them. A key problem to make adaptations work in practice is to understand the dynamics of such multi-agent systems, where the interface adapts to the user, while they in turn change their behavior in response to this adaptation. This is a new topic, follow us to learn more about it in the future.
Computational design of user interfaces
We develop optimization methods that can be used to automate (parts of) the design process of user interfaces. In addition to developing optimization models that capture the problem of UI design, we also focus on methods and tools that facilitate participatory optimizaton to enable stakeholders at all levels to take part in the design process. Interactive optimization tools can help them to learn about the problem and broadly explore possible solutions. They also make assumptions and trade-offs more explicit and help to objectively assess individual’s expectations and ideas.
See for example the following works:
- AZERTY Amélioré: Computational Design on a National Scale. In: Communications of the ACM, vol. 64, no. 2, 2021, ISSN: 0001-0782.
- Computational Support for Functionality Selection in Interaction Design. In: ACM Transaction on Computer-Human Interaction, vol. 24, no. 5, 2017, ISSN: 1073-0516.
- Assignment Problems for Optimizing Text Input. Aalto University, 2018.
- Towards Multi-Objective Optimization for UI Design. Workshop on Principles, Techniques and Perspectives on Optimization and HCI, CHI'15, Seoul, Korea, 2015.
Understanding and predicting user behavior
Our work builds on empirical studies to capture users’ behavior in interaction with different interfaces. This includes planning and conducting studies, analyzing data, and transforming empirical observations into predictive models using machine learning, statistical modeling, and other techniques. The results are actionable insights that can be used for optimization and adaptation or implemented as part of design tools. See for example the following projects:
- XRgonomics: Facilitating the Creation of Ergonomic 3D Interfaces. In: Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI'21), ACM, 2021.
- Complex Interaction as Emergent Behaviour: Simulating Mid-Air Virtual Keyboard Typing using Reinforcement Learning. IEEE Transactions on Visualization and Computer Graphics, IEEE, 2021.
- How We Type: Movement Strategies and Performance in Everyday Typing. In: SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, NY, US, 2016, ISBN: 978-1-4503-3362-7.
- Investigating the Dexterity of Multi-Finger Input for Mid-Air Text Entry. In: SIGCHI Conference on Human Factors in Computing Systems, ACM, New York, New York, USA, 2015, ISBN: 9781450331456.